Communication Efficient Decentralized Learning Over Bipartite Graphs

In this paper, we propose a communication-efficiently decentralized machine learning framework that solves consensus optimization problem defined over network of inter-connected workers. The proposed algorithm, Censored and Quantized Generalized GADMM (CQ-GGADMM), leverages the worker grouping ideas Group Alternating Direction Method Multipliers (GADMM), pushes frontier in communication efficiency by extending its applicability to generalized topologies, while incorporating link censoring for negligible updates after quantization. We theoretically prove CQ-GGADMM achieves linear convergence rate when local objective functions are strongly convex under some mild assumptions. Numerical simulations corroborate exhibits higher terms number rounds transmit energy consumption without compromising accuracy speed, compared censored ADMM, method GADMM.